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This study was' undertaken to determine the factor 

re of patient j^anagement problems (PMPs) and to 

r ^uch factors^ are stable for different ^9Uj)s taking 

tion and for the sajie group over time. PMPs attempt • 

written form, the pr;0C9ss that a physician goes 

ing a patient. Two - examinatio^ns were administered to 

al students, the first du^ring^^^bei^ junior year and 

g their senior yea:r. The seconcl test was also ' 

a. second class of' students during their junior year. 

results indicated t.here are two components to medical 

as measured by BHPs^-data gathering arid management. 

e stable over groups and over time. (Author/MV) 
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r-i A FACTOR ANALYTIC STUDY OF BRANCHING PATIENT MANAGEMENT PROBLEMS 

Dorthea H. Juul, Michael J, Noe, Ph.D., and Rene L. Nerenberg 

K 

Branching patient management problems (PMPs) as, described by McGuire, - 
Solomon, and BaShook (1) were developed to simulate on paiper the physician's 
encounter with a pat4ent and have been widely used to assess, medical problem- 
solving ability^Thts-stfldy was->indertaken to further examine the nature- 
the facto.rs_under1ying performance on PMPs and to determine whether such 
fattors are stable for different groups taking the same examirratlon arid for 
the same* group over time. 

LITERATURE REVIEW * ' • 

I" 3 Study undertaken at-the University of IlTinois College of Medicine 
to analyze performance on, PMPs, the authors (2) concluded that medical pro- 
bl em-solving was highly donterit specific because there was a great deal of 
intra-individual. variability in performance within and across different PMPs. 
Results of the Michigan State University Medical Inquiry Project (3) also 
indicated that there was a lack of consistency in performance aqross problems. 

' Bashook ('4) argues that this variability in performance is because there 
are different domains of clinical problem-solving. A domain is defined by 
J ^ ^^^^^ components-stage of the problem-solving process (sensing, defining, 

■ <o —— — — ■ • - , 
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reso-lvihg), clinical discipline, and the context of care (e.g., ernergency', 
acute, 'chronic, health maintenance). PMPs often 'focus on a single domain: 
hence, performance is not generalizable from one' problem to the next. 

However, Donnelly*, et al. (5) demonstrated that f.ere was some consis- 
tency fn performance across .problems. Ten PMPs and an average across all 
problems were 'individually factor analyzed, and two factors emerged— informa- 
tion gathering and decision making, the two factors were not highly related 
within problems or across problems. The high reliabilities of .the information- 
gathering variables and the low reliabilities of the decision making variables 
led the authors to conclude that information gathering is a general ability 
and that decision making is content specific because it varies from problem 
to problem. 

The research, although limited, does suggest that PMPs- do not measure a 
general medical problem-solving ability. There is some evidence that perfor- . 
mance may be content specific, and one study indicates there may be two com- 
ponents involved— information gathering and decision making. 
METHOD ^ • . — ■ ■ ■ 

•As indicated earlier, PMPs are an attempt to simulate in written fofin ; 
the process .that a physician goes through in managing a patient. They gener- 
ally consist of a short introduction to the patient followed by a series of 
sections devoted to gathering' history and physical e^am data-, ordering diag- 
nostic procedures, and treating the patient. Within each section the examfnee 
selects from a list of options those which he 'feels are appropriate. He' re- 
cords his decisions by erasing the opaque "overlay or developing the latent' 
image with a special pen to reveal the outcome of his choices. The responses^ 



are presented in as reaJistic a manner as possible. History questions are 

answered in terms the patient would be expected to use. ^Lf-x--rays7~EKGs^ 

?nd the like are' ordered, appropriate photographic reproductfons.*'are pro-. 

vided. fn additfon, no interpretation of results is given unless stich infor- 

mation is' normal lly provided or consultation is specifically requested. 

Each examinee determines the order in" which he goes thrpugh the problem 
, . ' •' . 

based on his judgment of optimal management of the patient. Therefore, not 

all examinees are exposed to the same se.ctions of the, problem nor tb the 
same sequence of sections. For example, selecting an incorrect medication, 
might cause the patient to" have an adverse reaction, and the .examinee must 
then deal with that complication. Selection of the proper medication, on" 
the other hand, would have led to satisfactory recovery. 

Two examinations that included PMPs were administered to 191' University 
of Illinois College of Medicine students (Class I), one during their junior 
year and the other^ during their senior year, as part df the ongoing appraisal 
program, The first exarii '(Exam I), "taken when the studen.ts were juniors, con- 
tained 24 PMPs, arid the second (Exam II), taken, when they were seniors," hac( 
26.. Exam II was also administere^d to a different group of 2i4 junior students 
(Class II). Both tests were designed to assess clinicdl competence in a num- 
ber of different disciplines, settings (e.x|. , emergency room, outpatient 
clinic, private office, hospital service), and types of problems ^.g. emer- 
gency, chronic). Although the specific content of the problems varied from* 

t 

year to year, the patien>ts had relatively commqn problems that third- and 
fourth year medical students were expected t^i'be able to manage. 

Becaus'e earlier research indicated that there was a great deal of 



vanabm-ty in individuaj performance apross problems i ij: was concluded lhat 

it would not be fruitful to analyze each | problem separately. Rather, it was 

hypothesized that given a sufficiently large and varied sample of PMPs, 
' ■ - • . , ' ■ i \ 

examinees would exhibit consistent patterns of performance over time. It • 

was further hypothesized that"differen| gioups of examipees would exhibit 
similar patterns of .perforipance on the sajije exam. 

In order to analyze performance across problems each of the items in the 

exams v«s classified into one of the six fallowing categories. * 

'* II ' ' 

History : Information gained from^ patient histoid • ' ' 

•PhZilcai: Information ga.i.ned front phVsical exam of the. patient 
Diagnostic procedures: Information gained from laboratory tests, x-rays, 
EKGs, etc. I . ■ 

Pathwaii.: Decision points at which thfe next stfige in management is .'. 
selected ' | ' . - , . 

• Treatment ; The care givep the patient; includes medications, operations, 
counseling, etc. | • 

, Diagnosis r Specific, identification of the patient's problem(s)' " 
Table 1 contains a percentage breakdown bi^ .category of the two ex;'V^ 



Insert Table 1 about here 



Each of the items was assigned a weight on a nine-point -scale ranging ' 
from +8 (clearly indicated) to -8 (contrail ndicated) by an interdisciplinary. 
coRjjnittee of medical school faculty.. Sjoi^es for the six- categories were 
computed for each student. These- scores were the algebraic, sum of the weights 
of the positive items and the negative 'itehis selected in each category across 



s 



all problems divided by the maximum number of points possible in that 



r ♦ 



category. 

Pn*ncipal-factor"analyses wi^'th iteration wer^e perfornjed separately on * 
the correlation matrices, of the three'exam administrations to identify fac- 
tors underlying performance on PMPs>. Initial estimates of communalities 
were the squared multiple correlations of each variable with the remaining 
five variables, and the significant -factors of the three analyses were rota- 
ted obliquely by t-he direct oblimin rotation method. In order to determine* 
if performance was similar for different groups , taking the same exam, the 
factor patterns for the Class I seniors and the Class II juniors were com- ' 
P^d. In-order to determine stability over time, a principal -factor analysis 
^was pe^ormed on' the correlation matrix of the 12 variables from the 6xams • 
'admin isteV'e^ to Class,! in the jtinior and senior years. 
RESULTS . v-X" ' ■ • ' ' * . 

• The results of the separate factor analys"es performed on the correlation 
.matrices of the three exam ^administrations are shown in Table. 2. Loadings 
(factor pattern- coefficients) greater than'.30 are underlined for emphasis. 
For each, of the three analyses two distinct factors emerged that were asso- 
ciated with principal components whose eigenvalues were greater than one and 
^ that. jointly accounted for 71% or 72% of the total variance in-the initial 
principal components 'analyses. History, physical, and diagnostic procedures 
loaded on one factor which was labelled "data gathering"-. Pathway, treat- 
ment, diagnosis, and diagnostic procedures Igaded on the second factor whicti 
was labelled "management". It should be noted that diagnostic procedures 
/loaded on both factors. The correlations between the two factors were .42 
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for the Class^iI>.jum'ors who took Exam I, .53 for the Class Iisem'ors who' 

„ • ■ \ ■ • , 

took Exam II, and .44 for the Class II juniors who took -Exam II. 

' Insert Tjable 2\Bout here • 

« ' - 

t * 

Inspection of the .Class II junior factor pattern revealed that it was 
very similar to the Class I senior factor pattern, and further rotation to 
maximize congruence was not necessary. The" similarity of the matrices indi- 
cated that the factors. wer^^stable for different groups taking the same exam. 

The results of the factor analysis performed on the correlation nfetrix 
of the 12 variables from-the two exams Ih^t were administered to Class I 
are shown i-n Table 3. Four factors emerged whose initial eigenvalues accoun- 
ted for 72% of the total variance. . These were junior and senior data gather- 
ing and jtmior and senior management. Junior diagnostic procedures loaded 
on junior data gathering and junior management. Senior diagnostic proce- 
dures loaded on senior data gathering and senior management, although less ■ 
heavily on the man^^ement factor.' <» ' " ' ' , 

The highest factor correlations in Table 3 were between jumor and senior 
data gathering (.49), junior and senior management (.60), .and s'enior data 
gathering 'and senior. management (,.51). 

ins§rt Table 3 about here 

DISCUSSION ■ . , \ 

The results of this study indicate that there are two components to 
medical problem-solving as measured by PMPs— skill in data gathering and/ 
skill in management. Both factors were stable for different groups who 



tcok th«2 same exam and over time for one group_ who took two different exams. 
. The stability of the factors over ttme is further emphasized by the relatively 
high correlations between junior and senior data gathering (.49) and junior 

and senior management (.60) as reported in Table 3. 1 

Donnelly and* his colleagues identified the same two dimensions. However, 
in the present study the data gathering and management factors* within each 
exam were positively correlated sugges^-ing that the factors are not indepen- 
.dent.' ' This finding is in contrast to the Donnelly sturdy which concluded on 
the basis of low obtained canonical correlations between the variables* com- 
prising aa'ch factor that the factors were unrelated. It seems likely that 
. appropriate management of a patient is to s certain exte^it dependent on the ' 
.adequacy of the data base developed -from the history, physical, and diagnostic' 
procecfures. However, an examinee might ask all the rjght questions -but not 
be able to- integrate the information to arrive at''an apijropr.i,ate resolution 
• -to the problem, or he might arrive at the appropriate resolution .from a 
sketchy data tase. • . " . . ' " ' ■ 



The selection of diagnostic procedures appears to be a significant point 
in this process of integration. Diagnostic procedures- may have loaded on both 
factors because that is the point at which information .obtained from the his- 



'tory and physical must be integrated in order to select correct diagnostic 
. procedures. The results of the procedures then influence the subsequent 
nanagement decisions made by the examinee. 

The i^esults of this study also provide spme evidence of construct 
validity of PMPs. The correlation between data gathering and management 
factors as reported in Table 2 was higher for the seniors (.53) than for the 
, juniors (.42 for. the group that took Exam I and .44 for the group that took 
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Exam II). This suggests that students at the senior level who usually 
have had more clinical experience are better able to/apply data gathering 
information to management decisions. • * 

The mean scores and standard deviation^ for the six categories for the 
juniors and.^seniors who took-'Exam II appear in Table 4.. A one-way multi- 
variate analysis of variance ^gerformed on the means of the six variables.,- 
-for the tvvo classes produced a significant mu,l ti variate- F-ratio (F = 13.06;- 
df = 6,398; p < .001), indicating that test performance is related to year 
in medical schbol-seniors performed .significantly better than juniors". ' It. 
might be argued that the improvement was due to increased famiVi^rity with 
the testing technique. However,' the students are given- careful .instructions 
prior to the exam, and many have encountered PMPs on previous exams and " 
through using- such texts as Cli'nical Simulations (6) which contai^ PMPs. 

**» ; .> 

' ^ • Insert Table 4 about here , ^^ilifc. 

In supnary, the resi/lts of this study support the findings of a previous 
study that ^suggested PMPs measure two factors-data gathering and .management- 
and tha"t at least two subscores are necessary to describe "perfoniiance-: — In — 
addition., it was found that these two. factors were stable across time and , 
across groups given a large and varied sample of PMPs. It is advisable, 
therefore, to sample broadly when using PMPs in order to obtain a stable 
measure of an indivic^ual 's ability. Evidence supporting the construct valid- 
ity of written simulations also emerged. 



TABLE 1 : . ' . 

- COMPOSITION OF EXAMS I AND II By'CATEGORV-- 

9 

■ . . EXAM I 

Disciplines represented: Medicine, Neof'STogy, ObstetriGs-Gynecology. 
Pedia4:ncs, Psycfviatry, Surgery (24 problems total) : 



CATEGORY PERCENT OF EXAM 

• > 

History 10.6 

Physical . ' ' " -9.5 

Diagnostic Procedures 27.4 

Pathway ^ ' 16.6 , - ' 

Treatment _ ^ 28.6 

Diagnosis * ' 7^2 



100.0% 

Administered to Class. I as juniors'(N = 191) 

EXAM II 



Disciplines represented: Medicine, Neurology, Obstetrics-Gynecology, • 
Otolaryngology, Pediatrics, Psychiatry, Surgery (26 problems total) 

CATEGORY ' - PERCENT OF EXAM ' ' ' ' . 



History . ~ 13.6 

Physical '11.2 

Diagnostic Procedures 28.4 

Pathway . . ' 13.0 

- Treatment ' * 30.2 

Diagnosis 3.6 



tr 100.0%. 



Administered to Class I as seniors (N = 191) and Class II as juniors (N 
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DIRECT OBLIMIN FACTOR PATTERNS 'FOR CLASSES I AND II' 



Factor 

Correlations 



CLASS I, JUNIOR YEAR 
EXAM I 
(N=191) 



CLAS5- I, SENIOR YEAR 
EXAM II ■ 
" ■■ (N=I91). 



CLASS II, JUNIOR YEAR 
- EXAM II 



✓ 

CAT£GORY ■ 


FACTOR -I 
DATA 
GATHERING 


FACTOR II 
MANAGE- 
MENT . 


. FACTOR I 
. ^ ^ DATA 

G :hering - 


factor ii 
mAnage- 

• MENT ^ 


FACTOR I 
DATA 
GATHERING 


, t 

FACTOR. I 
,' ' MANAGE- 
' MENT 


History 


" 0.63 - 


- - * 
0.12 , 


6.85 


6.04 . 


. 'o.85 


j. 0.00 


Physical 


0.93 


-0.13 


6.98* 


.-0.09 . 


0.^^97 


-0.09" 


^ Diagnostic 

Pro^cediires 
♦ 


0.47 


• » 

0.46 


. 0.51 


0.38- 


"0.35 


0.32 


Pathway 


.0.24 ' 


■ '6^56, . 


"o.io 


' 0.57 


' 0.'03 . 


0.70 


Treatment 


-0.01 


0.78 


-o.oi' ■ 


.0.65 . 


'0.07 


'0.63 


'Diagnosis 


-0.11 


0.75 


-0.05. 


0.71 . 


: r0..07. . 


0'.68 , 








# * * 









■'0.42 



0.53 



' '0.44 ■ P 
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DIRECT OBLIMIN FACTOR PAHERN FOR^CLASS I ON EMHS" I AND IT 



"CATf&ORY' 



FACTOR r 
OUNIOR DATA 
GATHERING 



FACTOR II . 
' SENIOR DATA 
GATHERING * 



FACTOR III ■ 
: JUNIOR 
MANAGEMENT 



FACTOR IV 
' SENIOR 
MANAGEMENT 



Jr." Hi-story 


• ^. 0.60 • 


■ 0.06 " ' 


• ' 0.13 


•Sr. "Histbry , 
Jr. • Physical; 


' ' *0.08 

a fx 

0.80. 


' V • 0.12 


, -iO.OS 
, -0.10 V 


Sr. Physical ' ' 


0.04 " 


• • \ • •* . 


o.do 


;Jr. Diagnpst'ic 
V Procedures ' - 

0 


0.40 


..•••\ ; 

' ^ 0.14-, 


> 

0.48 ■ 


Sr. Diagnostic- 
• Procedures 


' -0.04 


0.S9 


? 0.10 


Jr. ^Pathway ' 


'0.27« 


. , -0.^06 


. ^0.53 


Sr.* Pathway 


0.Q6 . 


0.09. 

1 


1 

. -0.02 


„Jr. Treatment 


; 0.04 


% 

r -0.tf2 


0.78 


Sr.* Treatment 


-0.13 


. • ■ 0.07 . 


0.30- 


Jr." Diagnosis 


''-6.14 ■ 


O.Qfi 


* 

.■ ■ 0.69 


Sr. Qiagnosis 


'J3.05 . 


-0.04' 


. -q.oi 



0.05 
. .0.05 
0.08* ' 
-p. 11 •■ 

0.00 .. 

0.28 • 
" 0.10 . 

T 

0.57 
-0.01 . 
• 0.45 
, 0.07 ' 

■0.71 .' 



.FACTOR f 
FACTOR II 

FACTOR in / 

FACTOR *IV . 

r 

J' V 



• Factor correlations 

..FACTOR l'-^ 'FACTOR II 

1.00. * 0^9 

. ' • -i.oo 



FjjCTOR lYl 
- 0.29 
• 0.33'- 
1.00 



/ACTOR IV 
0.30 

0:51 

. * 0.60 
- '«L06' 



0 * 



TABLE 4 



CLASS I AND CLASS II SCORES ON EXAM II 



, CLASS 1, SEWIOR YEAR 
(N=,191) 



. CLASS II, j'UNipR YEAR 
• {N=214)\ ■ 



1^ 



CATEGORY ' 

History 

Physiccil 

DLj a gnostic 
Procedures 

Pathway ' * 

'Treatment 

Diagnosis* 



f 



MEAN 



67.75 
73.78 

"57.35 

'75.73 
--3.8. -27. 
56.91 



. STANDA'RD 
DEVIATION 

10.26 

• 9.93 

7.35 ' 

- 8.83 ~ 
'7.79," 
8.53;.. 



MEAN 



61.93 
68.39 

52.26 

(59.53 

'32.94' 

51.-35 



STANDARD 
DEVIATION- 

11.06 

10.87 

« 

8.26 

' .10,07 
'7.50 
^.'55 
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